This invention relates generally to methods and systems for testing humans and systems and the subsequent classification of humans into knowledge states and systems (including human systems) into functionality states. More specifically, the invention relates to computer-implemented testing and classification systems.
The process of testing and classification requires meaningful and accurate representations of the subject domains in terms of domain states. A domain state that a test subject is in is determined by sequentially administering to the test subject test items involving different aspects of the subject domain. The responses of the test subject to the test items determines the state of the test subject in the subject domain.
The implementation of such a testing and classification process by means of a computer has the potential of providing an efficient and effective means for identifying the remedial actions required to bring the test subject to a higher level of knowledge or functionality.
The partially ordered set ("poset") is a natural model for the cognitive and functionality domains. Two states i and j in a poset model S may be related to each other in the following manner. If a test subject in state i can respond positively to all the test items to which a test subject in state j can, but a test subject in state j may not be able to respond positively to all the test items to which a test subject in state i can, we say that i contains j and denote this by the expression i.gtoreq.j. Note that a positive response on any item should provide at least as much evidence for the test subject being in state i as in state j. Thus, the domain states are partially ordered by the binary "i contains j" relation. Note that the cognitive level or the functionality level of a test subject in state i is equal to or higher than that of a test subject in state j. Similarly, the cognitive level or the functionality level of a test subject in state j is equal to or lower than that of a test subject in state i. Accordingly, state i is said to be equal to or higher than state j and state j is said to be equal to or lower than state i.
Poset models in an educational context have been proposed before. However, they have either been Boolean lattices or posets closed under union in the sense that the union of any two members of the poset is also in the poset. This restriction is undesirable in that it leads to models that can be quite large. For example, allowing the number of test items to define the model can lead to models with as many as 2.sup.N possible states where N is equal to the number of test items. With this approach the responses to the test items permits immediate classification with very little analysis. However, having such overly large models ultimately results in poor classification performance.
When sequential item selection rules have been used in classifying states in a poset, the approach has not been accomplished in a decision-theoretic context. Consequently, there was no assurance that the classification process would converge rapidly nor, in fact, that it would converge at all.
There is a need for a testing and classification system which is based on sound scientific and mathematical principles and which, as a result, can accurately and efficiently determine the domain states of humans and systems. It is reasonable to base such a system on poset models, but it should be possible to use general, even non-finite posets rather than the specialized posets that are typical of present-day systems. It is important that model selection and fitting for any particular domain be based on appropriate analysis rather than simply a result of the choice of test items. Similarly, the selection of test items should be based on appropriate analysis with reference to the domain model rather than being a more-or-less ad hoc process that ultimately gives birth to its own domain model.